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INTRODUCTION 


The  Institute  for  Human  and  Machine  Cognition  (IHMC)  is  pleased  to  submit  a  report  of 
progress  on  the  project  Human  Systems  Technology  for  the  2005  fiscal  year.  This  fiscal 
year  work  actually  began  on  May  1,  2005  and  was  completed  on  July  31,  2006.  Also 
called  Human-Centered  Computing,  this  multidisciplinary  field  exploits  advances  in 
cognitive  research  together  with  those  in  computer  science  and  related  areas  to  optimize 
the  cognitive,  perceptual,  and/or  physical  performance  of  experts  and  expert  teams  and 
the  information  systems  that  support  them.  The  work  reported  herein  extends  that 
conducted  in  the  previous  increment  of  ONR  funding  for  Human  Systems  Technology. 
The  FY  2005  work  focused  in  particular  on  (1)  psychophysical  studies  of  tactile 
interfaces  for  sensory  substitution  or  augmentation  in  complex  tasks;  (2)  continued 
development  of  advanced  algorithms  for  Knowledge  Discovery  and  Data  Mining  (KDD) 
from  large  data  sets  and  associated  investigations  of  displays  and  training  principles  to 
improve  human  abilities  to  recognize  causal  relationships;  (3)  fundamental  studies  of  the 
trustworthiness  of  agents  through  a  theory  of  adjustable  autonomy;  and  (4)  fundamental 
studies  of  exoskeletons  for  human  performance  enhancement.  The  work  in  each  of  these 
areas  reflects  the  growing  appreciation  for  the  enormous  potential  that  information 
technology  has  to  leverage  and  amplify  human  capabilities.  Realization  of  this  potential 
requires  a  deep  understanding  of  human  cognition,  perception,  and/or  locomotion;  the 
relevant  areas  of  computer  science;  and  the  nature  of  the  human  activity  to  be  enabled. 
This  inherently  multidisciplinary  approach  to  realizing  the  full  potential  of  emerging 
information  technology  capabilities  is  at  the  heart  of  the  Institute  for  Human  and  Machine 
Cognition  (IHMC)  and  the  research  conducted  during  FY  2005. 

In  what  follows  the  progress  made  in  each  of  these  areas  in  FY  2005  is  described.  In  each 
area  we  shall  describe  the  objective  of  the  work,  the  approach  taken,  a  concise  progress 
statement,  an  expanded  description  of  progress,  and,  where  applicable,  lists  of 
publications,  awards,  patents,  and  technology  transfer  activities. 


PSYCHOPHYSICAL  STUDIES  OF  TACTILE  INTERFACES  FOR 
SENSORY  SUBSTITUTION  OR  AUGMENTATION  IN  COMPLEX 
TASKS 

Scientific  and  Technical  Objectives 

Tactile  interfaces  have  been  developed  for  sensory  substitution  applications  such  as  aids 
for  the  vision  or  hearing  impaired.  Tactile  interfaces  have  also  been  developed  for 
sensory  augmentation  in  complex  dynamic  environments  such  as  aviation  applications.  In 
the  former  application  great  effort  is  taken  to  represent  the  sensory  information  in  as  high 
resolution  as  possible  (e.g.,  to  let  the  operator  “see”  detail  with  the  sense  of  touch.)  The 
latter  application  seeks  to  reduce  the  resolution  to  the  most  basic  useful  tactile 
representation  of  particular  sensor  information  in  order  to  reduce  the  cognitive  workload 
of  understanding  relevant  information.  In  their  current  incarnations,  both  methods  operate 


without  competition  for  sensory  or  cognitive  assets,  as  little  other  tactile  information  is 
useful  in  their  respective  operational  environments. 

In  1995,  the  Naval  Aerospace  Medical  Research  Laboratory  (NAMRL)  and  the  Florida 
Institute  for  Human  and  Machine  Cognition  (IHMC)  began  collaborating  on  flight-testing 
NAMRL’s  Tactile  Situation  Awareness  System  (TSAS).  Demonstrations  have  been 
conducted  with  a  suit  that  provides  pilots  with  spatial  orientation  information  via  tactile 
stimulation  in  order  to  reduce  the  number  of  SD  accidents  occurring  with  pilots.  The  suit 
was  successfully  flight  tested  on  a  blind  folded  pilot  of  a  fixed-wing  (T-34)  aircraft  and 
on  several  blind  folded  pilots  of  a  rotary-wing  (UH-60)  aircraft.  The  UH-60  pilots 
successfully  flew  basic  instrument  tasks  including  unusual  attitude  recovery  and  in  the  T- 
34,  the  pilot  successfully  flew  acrobatic  flight  maneuvers  in  addition  to  instrument  flight 
tasks.  Subsequent  work  has  shown  the  effectiveness  of  this  approach  to  unmanned  aerial 
vehicle  (UAV)  control. 

High-resolution  tactile  displays  can  only  be  effective  if  the  areas  they  contact  are  highly 
innervated.  Mapping  of  sensitivities  has  shown  that  the  highest  resolution  of  innervation 
occurs  in  the  hands,  face  and  tongue.  In  the  aviation  environment  the  tongue  makes  for  an 
ideal  location  for  a  high-resolution  tactile  interface.  Unlike  the  hands  it  is  not  in  active 
use,  except  during  verbal  communications,  and  it  has  no  keratinized  skin  layer,  allowing 
for  more  efficient  transduction  of  the  tactile  signal.  We  proposed  to  use  our  tongue  based 
tactile  interface  to  provide  higher  resolution  information  in  concert  with  the  lower 
resolution  information  of  TSAS.  This  would  represent  a  confluence  of  two  tracks  of 
thought  on  tactile  displays,  one  of  providing  sensory  augmentation  displaying  intuitive, 
dimensional/  directional  information  (as  with  TSAS)  and  the  other  of  sensory  substitution 
where  information  normally  perceived  via  the  eyes  or  ears  is  perceived  through  a  tactile 
interface.  The  goal  of  this  integration  will  be  to  create  a  tactile  interface  that  operates,  in 
3D,  in  a  manner  similar  to  the  eye,  with  low  resolution  “peripheral”  sensation  provided 
on  the  torso  by  TSAS  and  high  resolution  “foveal”  sensation  provided  on  the  tongue, 
forming  a  “haptic  retina”.  The  resultant  multi-faceted  tactile  display  is  being  tested  in 
concert  with  visual  and  audio  displays  using  a  fixed  base  simulator  at  IHMC. 

Approach 

We  evaluated  the  effectiveness  of  the  TSAS  interface  and  developed  the  software 
infrastructure  necessary  to  manage  the  information  flow  across  multi-sensory  (e.g., 
visual,  auditory  and  tactile),  multi-resolution  (i.e.,  low  and  high)  displays.  We  performed 
a  series  of  experiments  to  test,  evaluate  and  refine  the  effectiveness  of  multi-modal 
displays  in  dynamic  environments. 

Concise  Progress  Statement 

The  effectiveness  of  the  TSAS  torso  tactile  interface  for  learning  a  complex  dynamic  task 
was  tested  with  experienced  active  duty  Air  Force  test  pilots  (n-14)  and  novices  (n=27). 
Both  populations  used  the  fixed  base  simulator  at  IHMC  running  a  dynamically  accurate 
simulation  of  the  US  Space  Transportation  System  (“Space  Shuttle”)  and  both  were  able 


to  perform  the  space  shuttle  approach  and  landing  skills  more  effectively  when  using 
TSAS  than  when  using  only  the  visuals  provided  by  the  simulation.  This  task  requires  a 
dead-stick  landing  with  an  aerospace  craft  that  has  poor  aerodynamic  handling 
characteristics.  The  frequency  response  to  a  moving  target  with  multisensory  displays 
was  evaluated  and  IHMCs  Adaptive  Multiagent  Integration  (AMI)  architecture  was 
extended  and  leveraged  to  support  multimodal  displays,  intelligent  information 
scheduling  and  to  provide  the  basis  for  the  proposed  “haptic  retina”. 

Expanded  Description  of  Accomplishments 

We  integrated  our  Adaptive  Multiagent  Integration  (AMI)  architecture,  developed  under 
funding  from  DARPA,  with  our  tactile  interface  devices  (the  TSAS  torso  tactile  interface 
and  BrainPort™)  to  facilitate  integrated  testing.  Delivery  of  a  second  generation  version 
of  the  Wicab  BrainPort™  was  delayed  approximately  1  year  due  to  design  and 
manufacturing  issues  and  prevented  us  from  fully  testing  our  integrated  high  and  low 
resolution  tactile  display  concept.  However,  the  new  BrainPort™  has  been  integrated 
with  TSAS  and  initial  testing  of  the  combined  system  is  working  as  expected.  Custom 
torso  tactile  interface  hardware  fabricated  in  FY04  has  been  revised  to  improve  discrete 
control  of  the  torso  tactile  interface  and  to  enable  use  of  intelligent  automation  features 
made  available  through  integration  with  AMI.  Two  journal  articles  have  been  drafted 
describing  the  results  of  the  FY04  experiments.  A  single  article  will  be  submitted  that 
incorporates  the  findings  of  the  studies  conducted  during  FY05. 

The  pilot  studies  included  a  dynamic  tracking  task  based  on  the  Space  Shuttle  Heads  Up 
Display  (HUD)  approach  and  landing  deviation  display  using  the  poor  ergonomics  of  the 
shuttle  rotational  hand  controller  (RHC)  joystick.  In  the  first  condition,  the  guidance  and 
velocity  vector  icons  were  driven  in  a  pseudorandom  fashion  using  the  sum  of  5 
sinusoidal  waveforms  (0.04,  0.08,  0.12,  0.16  and  0.22  Hz)  in  the  X  and  Y  axes.  The 
subjects  were  instructed  to  use  the  rotational  hand  controller  joystick  to  keep  the  two 
icons  superimposed.  A  second  condition  used  surround  sound  (visual  screens  were  blank) 
to  provided  the  directional  error  between  the  two  data  points  (driven  by  the  same 
sinusoids,)  and  a  third  condition  provided  the  subjects  with  the  same  deviation  display 
using  TSAS.  Four  more  conditions  that  represented  combinations  of  visual,  audio  and 
tactile  stimuli  were  presented  in  a  counterbalanced  fashion.  Fourier  analysis  was 
performed  to  determine  the  gain  and  phase  for  each  display  modality  combination  at  the 
tested  stimulus  frequencies  (see  Figure  1). 

The  analysis  demonstrates  that  the  tracking  task  was  sufficiently  difficult,  even  at 
relatively  low  frequencies,  to  induce  large  pilot  induced  oscillations  (large  gains  with 
phase  shifts).  Given  that  the  task  was  based  on  a  visual  tracking  paradigm,  it  is  not 
surprising  to  see  that  the  visual-only  presentation  provided  the  best  response  in  gain  (both 
axes).  The  multi-modal  display  (visual+audio+tactile)  is  however  the  next  best  across  the 
majority  of  the  frequencies  tested.  It  is  obvious  from  the  data  that  the  high  frequencies  in 
this  task  were  too  fast  for  accurate  tracking.  Follow  up  studies  will  focus  on  the  range 
from  0.04  to  0.1  Hz  to  better  characterize  where  the  transition  to  an  over  compensation 
response  arises.  Additionally,  we  will  investigate  the  tracking  response  to  single 


frequency  sinusoids  to  verify  that  the  pseudorandom  nature  of  the  tracking  task  did  not 
adversely  affect  individual  performance. 
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Figure  1;  Bode  plots  for  the  seven  pilot  study  conditions.  Gain  of  1  and  phase  lead/lag  of 
0  degrees  are  ideal. 


For  our  evaluation  of  the  performance  of  novices  in  the  conduct  of  a  complex  task,  we 
completed  testing  of  27  subjects  in  a  simulated  instrument  landing  system  (ILS)  final 
approach  and  landing  in  a  fixed  base  simulator  using  the  NASA  Ascent-Entry 
Trainer/Cockpit  Avionics  Prototype  Environment  space  shuttle  simulation.  All  subjects 
were  non-pilots  and  had  never  performed  instrument  flight  or  experienced  ILS  training. 
Each  subject’s  performance  with  and  without  the  tactile  display  was  characterized  by  a 
NASA-JSC  derived  Justiz  Numeric  Measure  (JNM)  landing  score.  The  maximum 
possible  JNM  score  is  100  for  perfect  performance,  and  a  crash  results  in  a  score  of  zero. 
The  landing  score  or  JNM  was  significantly  improved  by  the  presence  of  the  tactile 
display.  For  all  subjects  the  JNM  was  46.2  with  the  tactile  display  compared  to  30.8 
without  it  (p<000).  In  addition,  the  control  group  crashed  62%  of  the  time  while  the 
tactile  group  only  crashed  37%  of  the  time  (p<016).  Work  to  combine  a  low-resolution 
tactile  display  (torso)  with  a  high  resolution  one  (tongue)  involved  developing  a  Java 
software  agent  interface  for  the  tongue  display  that  allows  single  or  multiple  tactile 
elements  to  be  activated  with  the  following  characteristics:  azimuth,  elevation  and  range 
(egocentric)  as  well  as  amplitude  and  “irritability”.  Additionally,  custom  hardware  has 
been  designed  and  fabricated  to  allow  more  discrete  control  of  the  torso  tactile  interface 
than  previously  possible. 


Analysis  of  the  JNM  landing  scores  supports  the  gender  differences  noted  by  Kennedy  et 
al,  (1996)  with  the  male  group  consistently  scoring  higher  (male  average=47.96, 
female=33.3 1,  p<000)  and  crashing  less  often  (male  average=28.6%,  female  average= 
47.7%,  p<  009).  Due  to  this  difference,  data  was  analyzed  by  gender  (male/female)  as 
well  as  by  group  (test/control). 


Mean  TSAS  ON=46.2,  TSAS  OFF=  30.8,  p<.000  Mean  TSAS  ON=0.37  ,  TSAS  OFF=  0.62,  p<016 


Mean  TSAS  ON=5L.3,  TSAS  OFF=44.6  ,  p<.003  Mean  TSAS  ON=0.15  ,  TSAS  OFF=  0.29,  p<020 


Figure  2:  Landing  score  and  number  of  crashes  by  trial  (or  block  of  trials)  for  TSAS  vs. 
control,  demonstrating  gender  difference. 

A  second  study  performed  with  the  same  simulator  with  active  duty  Air  Force  test  pilots 
as  subjects  (N=14)  who  flew  the  approach  and  landing  in  reduced  weather  conditions  and 
partial  head  up  display  (HUD)  failure  starting  from  50,000  ft  altitude  demonstrated  that 
JNM  landing  performance  scores  improved  when  tactile  cues  was  added  following  initial 


training  (p<  .01).  From  the  data  it  is  clear  that  TSAS  allows  novice  subjects  to  achieve  a 
higher  level  of  performance  when  learning  a  novel  complex  task.  This  supports  anecdotal 
evidence  from  prior  TSAS  studies  where  students  claimed  that  they  would  be  able  to 
learn  complex  flight  tasks  more  quickly  when  using  a  tactile  interface. 

Publications,  Awards,  and  Patents 

Peer-Reviewed  Journal  Articles 


Raj,  A.  K.,  Higgins,  J.,  Abraham,  L.  J.,  Carff,  R.  W.,  &  Rupert,  A.  H.  (in  preparation).  An 
application  of  the  tactile  situation  awareness  system  (TSAS)  to  the  learning  of  a  complex 
dynamic  task  -  1LS  approach  and  landing. 

Abstracts/Presentations/Posters/Conference 


Raj,  A.  K.,  Higgins,  J.,  Abraham,  L.  J.,  Carff,  R.  W.,  &  Rupert,  A.  H.  (2004).  An 
application  of  the  tactile  situation  awareness  system  (TSAS)  to  the  learning  of  a  complex 
dynamic  task  -  1LS  approach  and  landing,  Aerospace  Medical  Association  75th  Annual 
Scientific  Meeting,  Anchorage  AK. 

Olson,  J.  M.,  Raj,  A.  K.,  Thomas,  R.  E.  and  Davis,  J.  A.  (2004)  Tactile  Display  Landing 
Safety,  Situational  Awareness,  and  Workload  Reduction  Improvements  for  the  Space 
Shuttle.  Aerospace  Medical  Association  75th  Annual  Scientific  Meeting,  Anchorage  AK. 

Awards/Honors/Invention  Disclosure 


Aerospace  Medical  Association,  Space  Medicine  Branch  Young  Investigator  Award  (best 
paper):  Olson,  J.  M.,  Raj,  A.  K.,  Thomas,  R.  E.  and  Davis,  J.  A.  (2004)  Tactile  Display 
Landing  Safety,  Situational  Awareness,  and  Workload  Reduction  Improvements  for  the 
Space  Shuttle  (submitted  to  Aerospace  and  Environmental  Medicine). 


ADVANCED  ALGORITHMS  FOR  HUMAN  SYSTEMS 
TECHNOLOGY:  Knowledge  Discovery  and  Data  Mining  and  Human 
Causal  Learning 

Scientific  and  Technical  Objectives: 

Our  objective  for  this  project  was  to  expand  the  available  suite  of  algorithms  for  data 
understanding — including  procedures  for  classification  and  recognition,  and  procedures 
for  inferring  mechanisms  from  observed  data,  and  procedures  for  reasoning  with 
uncertainty— by  mathematical  work,  computational  implementations,  and  testing  on  real 
and  simulated  data.  To  these  ends,  we  developed  graphical  algorithms  for  classifying  a 
target  in  data  sets  with  very  large  numbers  of  variables  but  comparatively  few 
observations,  and  algorithms  for  identifying  command  and  control  sub-networks  within 


much  larger  communication  networks.  We  also  improved  both  foundations  and 
implementation  of  algorithms  for  correcting  noisy  data — data  polishing — and  for 
estimating  causal  relationships,  and  we  explored  the  structure  of  non-monotonic 
reasoning  with  probability  bounds. 

Approach 

Our  approach  has  been  to  focus  on  problems  that  are  potentially  pertinent  to  Navy  and 
other  Defense  Department  data  analysis  problems  but  are  conventionally  held  to  be  too 
difficult  for  a  principled  solution  to  be  possible.  For  example: 

•  Automated  specification  of  causal  relations  from  uncontrolled  samples 

•  Locating  a  minimally  sufficient  set  of  predictor  variables  for  a  target  variable 
when  the  number  of  variables  far  exceeds  the  sample  size. 

•  Both  of  the  above  when  the  data  are  from  distinct  data  sets  with  no  cases  in 
common  (or  no  identified  cases  in  common)  and  with  distinct  but  overlapping  sets 
of  variables. 

•  Finding  errors  in  data  without  presuming  a  model  of  the  data  generating  process 

•  Clustering  measured  variables  into  groups  sharing  an  unmeasured  common  cause, 
and  extracting  information  about  the  causal  relations  among  those  unmeasured 
variables. 

•  Extracting  command  and  control  networks  from  data  bases  of  communication 
logs,  without  knowledge  of  the  language  of  communication. 

In  each  case  we  have  reconceived  the  problem,  pursued  formal  work  on  algorithms, 
implemented  the  procedures,  and  tested  them  on  simulated,  and  where  available  to  us, 
appropriate  real  world  data. 

In  addition  we  have  considered  theoretical  problems  of  automated  reasoning,  based  on 
interval  estimates  of  probabilities  and  their  propagation  through  courses  of  inference. 
Finally,  we  have  attended  to  some  fundamental  issues  in  cognitive  psychology,  especially 
theories  of  human  categorization,  which  seem  pertinent  to  Defense  Department  problems. 

Concise  Progress  Summary 

Our  major  accomplishments  this  year  included  the  design,  implementation  in  JAVA,  and 
testing  of  an  algorithm  for  extracting  command,  control  and  reporting  networks  from 
communication  logs.  Other  major  work  completed  this  year  is  a  classification  using 
graphical  models  of  almost  all  current  psychological  theories  of  human  category 
judgment.  The  representation  enables  us  to  systematically  understand  logical  relations 
among  the  theories  and  to  devise  experiments  to  differentiate  among  them. 


Expanded  Description  of  Accomplishments 

One  way  of  viewing  the  work  on  communication  networks  is  as  a  classifier  algorithm  to 
solve  the  following  problem: 

Given  a  set  of  time-stamped  communications  that 

•  Identify  the  communicators 

•  Have  text  in  a  possibly  unknown  language 

•  Involve  lots  of  people,  mostly  of  no  interest 

•  Involve  a  small  proportion  of  conspirators 

Identify  the  conspirators  and  their  relationships  (command/reporting)  while  minimizing 
information  intrusion 

Conventional  social  network  analysis  only  provides  unhelpful  “hairballs”  such  as  the 
following,  taken  from  simulated  data  logs  from  a  scenario  created  for  the  Department  of 
Defense: 


\.wabiKial 


In  this  scenario,  only  66  of  about  1000  communicators  are  conspirators.  There  are  more 
than  2000  messages. 


Our  algorithms,  which  uses  only  time  ordering  and  probability  measures  extracted  from 
frequency  of  communication,  find  about  a  third  of  the  conspirators,  and  correctly  links 
them  in  sub  groups.  Slightly  more  than  50%  of  the  names  returned  are  false  positives. 


•  33%  of  known  bad  guys  identified  as  part  of  a  command  structure 

•  Green  =  known  leaders  Red  =  known  bad  guy.  Blue  =  others  mis-identified 
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Technology  Transfer 

A  course  was  given  in  the  summer  of  2006  to  Department  of  Defense  personnel  including 
details  of  the  algorithms  and  applications  outlined  here  on  extracting  network 


information.  A  JAVA  implementation  of  two  algorithms  is  being  made  available  to  an 
agency  of  the  DOD. 


TRUSTWORTHY  AGENTS  THROUGH  A  MORE  ADEQUATE 
THEORY  AND  IMPLEMENTATION  FOR  ADJUSTABLE 
AUTONOMY 

Scientific  and  Technical  Objectives 

Highly  effective  teams  must  appropriately  trade  off  authority  and  responsibility  among 
various  team  members  (humans,  robots,  software  agents)  as  they  coordinate  joint  activity. 
In  contrast  to  typical  adjustable  autonomy  mechanisms  that  rely  on  coarse-grained  mode 
setting,  we  aim  to  develop  mechanisms  that  will  allow  us  to  demonstrate  the  benefits  of 
‘disciplined  initiative,’  namely,  automatically  adjusting  autonomy  and  system  initiative 
along  multiple  fine-grained  policy-bounded  dimensions  (e.g.,  permissions,  obligations, 
capabilities)  to  minimize  the  hazards  of  rigid  “modes”  and  to  provide  greater  team 
resilience  for  contingencies  (e.g.,  a  partially-disabled  team  member). 

Approach 

To  support  adjustable  autonomy  in  our  teamwork-oriented  infrastructure,  we  are 
developing  a  component  named  Kaa  (KAoS  adjustable  autonomy).  Assistance  from  Kaa 
in  making  limited  adjustable  autonomy  decisions  might  typically  be  required  when  it  is 
anticipated  that  the  current  configuration  of  policies  has  led  to  or  is  likely  to  lead  to  poor 
performance  or  failure,  and  when  there  is  no  set  of  competent  and  authorized  humans 
available  to  make  the  decisions  themselves.  Ultimately,  it  is  a  matter  of  expected  utility: 
the  utility  of  making  some  kind  of  change  vs.  the  utility  of  maintaining  the  status  quo. 

Kaa  uses  decision-theoretic  algorithms  to  determine  what  if  any  changes  should  be  made 
in  agent  autonomy.  When  invoked,  Kaa  first  compares  the  utility  of  various  adjustment 
options  (e.g.,  increases  or  decreases  in  permissions  and  obligations,  acquisition  of 
capabilities,  proactive  changes  to  the  situation  to  allow  new  possibilities),  and  then — if  a 
change  in  the  status  quo  is  warranted — takes  action  to  implement  the  recommended 
alternative. 

Our  first  version  of  Kaa  used  static  influence  diagram  models  which  did  not  vary  across 
situation  contexts.  This  approach  was  inflexible  and  inefficient.  One  of  our  major  goals 
for  this  year’s  research  was  to  leverage  the  power  of  the  logical  representation  and 
reasoning  mechanisms  in  our  KAoS  policy  services  framework  to  construct  an 
appropriate  probabilistic  influence  diagram  model  for  the  situation  at  hand  on  the  fly, 
consistent  with  the  rationale  first  given  by  Szolovits  and  Pauker  in  1978: 

When...  complex  problems  need  to  be  addressed...  then...  probabilistic  models 
are  necessary.  The  essential  key  to  their  correct  use  is  that  they  must  be  applied  in 


a  limited  problem  domain  where  their  assumptions  can  be  accepted  with 
confidence.  Thus,  it  is  the  role  of  categorical  methods  to  discover  what  the  central 
problem  is  and  to  limit  it  as  strongly  as  possible;  only  then  are  probabilistic 
techniques  appropriate  for  its  solution. 


Compose  influence 
Diagram  network  and 
static  node  valuations 


Maximize  expected 
utility  taking  observed 
world  state  and  risk 
attitudes  into  account 


Determine  Value  of 
Information, 
Value  of  Control,  etc. 


Our  aim  in  extending  Kaa  in  this  fashion  (shown  in  diagrammatic  form  above)  is  to  move 
from  a  situation  where  a  different  point  solution  for  adjustable  autonomy  is  required  for 
each  application  toward  the  goal  where  we  can  create  and  validate  a  general  model  for 
adjustable  autonomy  that  can  work  in  tandem  with  KAoS  teamwork  policies  across  a 
wide  spectrum  of  domains.  Moving  beyond  the  application-specific  influence  diagrams 
we  have  constructed  for  the  initial  prototype,  we  propose  to  represent  the  specific 
implications  of  our  general  model  of  adjustable  autonomy  in  skeletal  knowledge  bases 
(influence  diagram  templates)  of  probabilistic  information,  alternatives,  and  preferences 
that  can  be  reused  for  particular  classes  of  decisions.  Portions  of  this  knowledge  base 
would  then  be  combined  with  application-specific  and  situation-specific  information, 
alternatives,  and  preferences  obtained  at  runtime. 

Concise  Progress  Statement 

We  developed  initial  KAoS  ontologies  and  reasoning  mechanisms  to  allow  us  to 
represent  and  construct  influence  diagram  models  on  the  fly.  In  addition  we  have  begun 
work  on  generic  ontologies  of  team  recovery  alternatives  and  of  regulatory  concepts  to 
help  generate  and  reason  about  candidate  actions  that  can  be  fed  to  the  influence  diagram 
model.  We  have  constructed  a  demonstration  of  this  capability  that  will  serve  as  the  basis 
for  additional  experimentation  in  the  context  of  a  large  human-robotic  teamwork  exercise 
in  the  first  half  of  2007. 


Expanded  Description  of  Accomplishments 

The  basic  operation  of  the  system  is  illustrated  below.  KAoS  maintains  situation 
awareness  by  subscribing  to  relevant  sensor  and  other  information  feeds  relevant  to  the 
joint  activity.  When  the  KAoS  becomes  aware  of  a  failed  or  potentially  failed  action. 


creates  an  influence  diagram  network  for  Kaa  populated  with  appropriate  parameters 
from  the  knowledge  base  and  the  observed  world  state.  The  decision-theoretic  engine  in 
Kaa  then  computes  the  expected  utility  of  various  alternatives  (e.g.,  modify  some  aspect 
of  the  situation,  add  capabilities,  lift  negative  authorizations,  renege  or  delegate 
obligations)  and  selects  the  best  alternative  consistent  with  situation  constraints.  Users 
can  specify  which  actions  Kaa  can  be  allowed  to  adjust  and  which  it  must  leave  to  human 
judgment. 


Define  Logical  Knowledge: 

•  Action  capability  depends  on 
resource  status 

•  Utility  is  the  function  of 
context-based  attributes 


- ►  Influence  Diagram  Engine 


Create  Network  topology 


Compute  the  utility  of 
each  policy  choice 


Best  choice  in  the  current  context 
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Technology  Transfer 

We  will  use  this  capability  as  part  of  an  ONR-funded  Coordinated  Operations  project  to 
demonstrate  multi  human-robot  coordination  in  a  reconnaissance  scenario  in  a  port 
facility.  The  Air  Force  Research  Laboratory  has  recently  funded  an  SBIR  to  allow  us  to 
mature  this  capability  in  the  context  of  risk-adaptive  access  control  in  the  Global 
Information  Grid.  Additionally,  the  Army  Research  Labs  has  expressed  interest  in  this 
capability  and  plans  to  make  use  of  it  in  a  Blue-force  tracking  scenario  in  2007. 


EXOSKELETONS  FOR  HUMAN  PERFORMANCE 
ENHANCEMENT 

Scientific  and  Technical  Objectives 

Our  objective  for  this  project  was  to  expand  our  previous  research  on  exoskeletons, 
including  improved  man-machine  attachments,  algorithms  for  human  intention  detection, 
control  algorithms  for  locomotion,  and  improved  actuation.  Towards  these  ends,  we 
designed  and  built  two  exoskeleton  prototypes:  The  PISCES  underwater  exoskeleton,  and 
a  Powered  Climbing  Suit. 

Approach 

The  goal  of  an  exoskeleton  is  to  provide  the  user  enhanced  performance  by  increasing  the 
wearer’s  strength,  endurance,  and/or  speed  enabling  them  to  perform  tasks  that  they 
previously  could  not  perform.  At  the  same  time,  the  exoskeleton  should  not  impede  the 
user’s  natural  motion,  nor  interfere  with  sensory  or  cognitive  capabilities.  The 


exoskeleton  should  provide  a  natural,  intuitive,  transparent  interface  such  that  the  user 
feels  as  if  the  exoskeleton  is  a  true  extension  of  his/her  body  rather  than  something  that 
the  user  is  driving.  These  considerations  were  all  taken  into  account  in  the  development 
of  the  two  exoskeletons. 

Concise  Progress  Summary 

We  focused  our  efforts  on  two  robotic  exoskeleton  projects:  one  for  swimming,  and  one 
for  climbing.  PISCES,  the  swimming  exoskeleton  was  constructed  and  tested  in  a  shallow 
pool.  The  fit  on  the  user  was  comfortable,  and  the  device  survived  being  submerged  in 
water.  In  terms  of  performance,  there  was  limited  enhancement  due  to  undersized 
actuators.  This  problem  is  being  corrected  with  an  identical  form  factor  actuator  but 
difference  electro-mechanical  properties.  The  other  exoskeleton  that  was  developed  was  a 
Powered  Climbing  Suit  prototype,  which  we  designed,  built,  and  tested.  The  interface 
was  a  simple  handgrip  that  user  pulled  up  or  push  down  on  to  raise  or  lower  himself. 
With  this  device  the  user  could  perform  one  handed  chin  ups  on  a  test  bar  with  very  little 
effort. 

Expanded  Description  of  Accomplishments 

PISCES,  an  underwater  swimming  exoskeleton,  was  designed  to  increase  the  speed  and 
endurance  of  a  human  swimming  underwater.  The  exoskeleton  has  been  tested  in  the 
water  for  fit,  comfort,  and  range  of  motion.  The  force  measurement  system,  the  electronic 
system,  and  the  actuators  have  been  tested  underwater  for  an  extended  period  of 
operation.  Unfortunately,  with  the  motors,  amplifiers,  and  gear  trains  that  we  selected,  the 
swimming  abilities  of  the  user  were  not  extended.  We  are  currently  waiting  for  new 
components  that  will  have  a  factor  of  3  power  increase,  which  should  be  sufficient  for 
capability  enhancement. 

A  climbing  suit  exoskeleton  was  designed,  built,  and  tested.  This  device  featured  a 
preliminary  user  handgrip  interface.  One  end  of  the  interface  attached  to  the  cable  of  the 
device  and  the  other  end  had  a  hook  that  was  attached  to  a  fixed  point  on  the  ceiling. 
Built  into  the  handgrip  interface  was  a  load  cell  that  measured  the  force  the  user  was 
applying.  We  tested  the  device  by  having  the  user  pull  and  push  on  the  handgrip  to  raise 
and  lower  himself. 


Figure  I :  The  picture  on  the  left  is  PISCES,  an 
underwater  exoskeleton.  The  picture  on  the  right  shows  a 
user  wearing  a  climbing  suit  exoskeleton  and  raising  and 
lowering  himself  using  very  little  force  with  his  right 
arm. 
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